Object localisation, dimensions estimation and tracking.

PhD Theses.Localising, estimating the physical properties of, and tracking objects from audio and video signals are the base for a large variety of applications such as surveillance, search and rescue, extraction of objects’ patterns and robotic applications. These tasks are challenging due to low signal-to-noise ratio, multiple moving objects, occlusions and changes in objects’ appearance. Moreover, these tasks become more challenging when real-time performance is required and when the sensor is mounted in a moving platform such as a robot, which introduces further problems due to potentially quick sensor motions and noisy observations. In this thesis, we consider algorithms for single and multiple object tracking from static microphones and cameras, and moving cameras without relying on additional sensors or making strong assumptions about the objects or the scene; and localisation and estimation of the 3D physical properties of unseen objects. We propose an online multi-object tracker that addresses noisy observations by exploiting the confidence on object observations and also addresses the challenges of object and camera motion by introducing a real-time object motion predictor that forecasts the future location of objects with uncalibrated cameras. The proposed method enables real-time tracking by avoiding computationally expensive labelling procedures such as clustering for data association. Moreover, we propose a novel multi-view algorithm for jointly localising and estimating the 3D physical properties of objects via semantic segmentation and projective geometry without the need to use additional sensors or markers. We validate the proposed methods in three standard benchmarks, two self-collected datasets, and two real robotic applications that involve an unmanned aerial vehicle and a robotic arm. Experimental results show that the proposed methods improve existing alternatives in terms of accuracy and speed